Designing and Evaluating Dashboards for Multi-agency Crisis Preparation: A Living Lab

Public organizations show growing interest in the development of dashboards that aid relief agency managers in crisis preparation. Yet, there is a dearth of research on the development of such dashboards. This paper discusses the experiences gained from a pioneering Living Lab on the development and evaluation of dashboards for assessing crisis preparedness. In order to evaluate and further improve dashboards, a two-day user-centered gaming simulation was organized with forty relief agency managers. A survey distributed amongst the managers indicates that they were satisfied with the dashboards and intend to use these in practice. However, the managers suggested that the formulation and clustering of the performance indicators requires better alignment with the context of use. One of the main findings is that the high level of uncertainty regarding the final set of performance indicators and the corresponding norms demands flexibility in the dashboard architecture beyond the evaluation stage.